Robust Average-Reward Markov Decision Processes

In robust Markov decision processes (MDPs), the uncertainty in transition kernel is addressed by finding a policy that optimizes worst-case performance over an set of MDPs. While much literature has focused on discounted MDPs, average-reward MDPs remain largely unexplored. this paper, we focus where goal to find average reward set. We first take approach approximates using prove value function converges as discount factor goes 1, and moreover when it large, any optimal MDP also average-reward. further design dynamic programming approach, theoretically characterize its convergence optimum. Then, investigate directly without intermediate step. derive Bellman equation for can be derived from solution, relative iteration algorithm provably finds or equivalently, policy.